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Without objective measures of occupational 
competence, it is not possible to ascertain whether the prospective 
teacher possesses the broadly-based level of competence essential for 
effective teaching. Each occupation contains elements which 
distinguish the competent tradesman (the author identifies seven) • 
Assurance is required that the level and range of occupational 
competence of the future teacher is adequate to meet youth and adult 
training needs, and \.here is convincing evidence that measures beyond 
the paper and pencil test are needed to evaluate the. skill phase of 
competence. A National Occupational Competency Testing Institute 
(NOCTI) has been established to assure effective continuing service 
to vocational education; the Educational Testing Service will 
administer and score the NOCTI examinations nationally, report the 
test results, carry on research, and develop additional tests. (The 
test development procedure, an evaluation of test results, a 
discussion of test use, and a description of the contents of an 
examination packet are included briefly in the document.) (A J) 
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NEED FOR OCCUPATIONAL COMPETENCY EVALUATION 

When vocational training became a school responsibility through 
the passage of the Smith-Hughes Act in 1917, th^ selection of occupa- 
tionally competent people for teaching became a ni\jor responsibility 
of teacher educators and vocational administrators. With the growth 
of vocational education the need for objective, valid, and reliable 
instruments for the evaluation of occupational competence has become 
more and more urgent* 

Whether vocational education relates to the preparation of youth, 
new workers or the retraining or upgrading of adults; whether the 
training Is carried on in educational institutions, industry, by unions 
or the military, there is a force common to all to make the potential 
manpower actual manpower. That force is the occupational competence of 
the teachers. 

NEED FOR OBJECTIVE MEASURES 

Without objective measures of occupational competence, it is not 
possible to ascertain whether the prospective teacher possesses the 
broadly-based level of competence essential for effective teaching. 

The search for such measures has been carried on for many years, 
but only sporadically. It has been well known that few states have the 
financial resources, facilities or the personnel to develop reliable and 
valid comprehensive examinations which can be efficiently administered 
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and provide test scores that can be readily interpreted. Yet is was 
also known that a number of states were and are now using occupational 
competency tests. 

A sti^ll and determined group of individuals continued their efforts 
over the years but each in his own location. The first breakthrough 
occurred when two workshops at Rutgers University strongly urged a 
national effort for the development of National Occupational Competency 
Evaluative Instruments. 

Under the leadership of Carl Schaefcr, Mel Barlov>j Thomas Olivo 
and Richard Nelson, a grant was obtained from the U.S. Office of Educa^ 
tion which resulted in the National Occupational Competency Testing 
Project. Its objective was threefold: 

1. To establish a Consortium of States for Occupational 
Competency Testing; 

2. Develop test procedures and develop tests in 
selected occupational areas and explore the possibility 
of testing on a national scale through a pilot testing 
program; and 

3. Explore the desirability of a National Occupational 
Competency Testing Institute for reproduction, 
distribution, scoring, statistical analysis, and 
reporting back to the states. 

THE SURVEY OF THE STATE OF THE ART 

As a first step a survey was undertaken on a national scale which 



covered industry, labor unions, the civil service, vocational divisions 
of state education departments, and the military* Included was a search 
of the literature. The most significant findings revealed extensive 
duplication of efforts and little opportunity for test evaluation and 
improvement*. 

DEFINITtOM OF OCCUPATIONAL COMPETENCY 

The State of the Art definitely provided the direction in which to 
proceed. The nature of the instrtiments to be developed depended upon 
what exactly was to be evaluated. Occupational competency is a variable 
which is difficult to define. It includes casks which are performed by 
tradesmen at various levels. It Involves information, judgement, facts 
and how to apply them to work situations and a wide variety of manipu- 
lative skills which are required by daily work. The key for determining 
occupational conpetency of future vocational teachers is competency at 
an acceptable level . Each occupation contains elements which distinguish 
the competent tradesman from the less competent. Among the more easily 
recognizable elements ar^ the following: 

1, He has coir^iiand of the manipulative skills of his 
occupation in terms of technique, quality of work, 
and speed of performance 

2, He is able to diagnose and correct troubles and 
solve problems typical of the kind encountered in 
the performance of his occupation 
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3. He can plan his work efficiently 

4* He has knowledge of technical facts, underlying 
scientific principles, and is able to apply this 
knowledge to specific work assignments 

5. He is able to make all computations necessary to 
his work 

6, He can interpret drawings, diagrams, specifications 
and other directions pertinent to his field. 

Broadly stated, an individual is considered competent when he can 
carry on the work of the occupation at or above a certain level and within 
a broad range of assignments. 

The key to occupational competency definition is the LEVEL at which 
the tradesman functions. The vocational program administrator and the 
industry the program serves must have assurance^ that the level and range 
of occupational competence of the future teacher is f.dequate to meet the 
full spectrum of youth and adult training needs, 
KINDS OF INSTRUMENTS NEEDED 

The dominant practice in nearly all foms of evaluation is the pencil 
and paper test. It is objective, can be administered easily, and the 
results can be scored and reported easily and rapidly. There are points 
of view which hold ithat occupational competency can be evaluated by this 
means . 

There is, however, convincing evidence that added measures are needed 
to evalut'ite the skill phase of occupational competence. In a study of norm 
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patterns applied to the selection o£ trade and technical teacher candidates, 
Rellly concluded that written examinations alone are a poor predictor of 
trade competence* Stulf » who conducted extensive work on performance 
measures for the Navy during World War II, concluded that performance and 
Improved shop grades were not closely related to written test grades. On 
the other hand, Hankin stated that the correlation should be high If the 
study population from which the scores are drawn represent a normal range 
of competence, from low to high* If such correlations are low on one or 
the other of the tests, usually the written test is weak. 

The notion that if there is a high correlation only the written test 
would be needed derives from other fields of testing such as mental ability; 
wb^re the intention is to Isolate Independent traits such as space percep- 
tion and verbal masoning* 

Im mnnnnrlnir the occupationally competent worker fli has both ty^B3 of 
ability and they are not independent. It's only by written and performance 
tests that the breadth of competency can be evaluated* 

On the basis of available research data, combined with fi^ld studies 
and results obtained from a ?ilot Testing program it became evident early 
in the Project, that occuP''itional competency evaluation by means of 
examinations required both written and performance tests. 
DETERMINATION OF TEST CONTENT 

Vocational educators have long ago established a valid and reliable 
procedure for determining the learning or Instructional content of an 
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occupation through occupational analysis. This method has served us well 
and proved to be, wd.th minor modification, a very practical way for deter- 
mining the content for occupational competency examinations. Such an 
analysis must provide the necessary Information for the written test and 
the skills to be evaluated by the performance test. 

As Indicated earlier, many Industrial occupations Involve clusters 
of jobs ranging from the specialist to the all around tradesman. The 
first step in this analysis was to establish job clusters representing 
several levels of competence; then examine each level for facts, information, 
judgments, computaMons, and kinds of directions a competent man applies. 
Similar determination was made for the manipulative skills, their frequency 
of use, degree of complexity and commonality throughout the job clusters. 

Sucli as analysis requires great care and must be mad'^* by individuals 
representing a broad spectrum of the occupation who are recognized for 
their competence. They must be assisted by teacher educators and/or 
test technicians with special skills in competency analysis. 
TEST DEVELOPMENT PROCEDURE 

While the analysis of the items which constitute occupational compe- 
tence is a formidable task it is really only one of the steps required for 
test development. Since it is nojt possible to Include all items of an 



analysis in either the written or performance test, careful planning must 
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regional preferences and practices,^the physical feaslbllty of perfor- 
mance demonstration. The survey of the State of the Art clearly 
established the need of a practical and reliable procedure which was 
applicable to all occupations. I^p^^ 

The following ten step procedure was followed with satisfying 
results: 

1« Establishment of an Advisory Committee consisting 
of representatives of industry and/or occupations 
to determine the levels of competence and overall 
test content* 

2» Designation of recognized and competent individuals 
and analysis specialists to conduct the occupational 
analysis. 

3. Designation of Individuals from the occupation, 
vocational teachers and test technicians to prepare 

. a test grid for adequate coverage of all areas of 
the occupation* 

4. Preparation of test items for written and performance 
tests by individuals from industry, test technician 
and vocational teachers, 

5. Organization of test items into a first draft test 
by test technician. 

6. Test administration to one or several outstanding 
craftsmen in tlie occupation for review and comment. 
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7. Test revision according to responses and recommendations 

8* Test submitted to professional testing organization 
such as Educational Testing Service, Psychological 
Corporation for review and revision. t 

9. Pilot administration of tests to obtain data and^est 
administrative procedures 4 

10. Scoring, reporting to candidates — followed by 

statistical analysis, item analysis and test revision. 

NUMBER OF TESTS COMPLETEP 

Even a casual review of this procedure indicates that most states 
and/or Institutions do not have the resources — personnel, finances, 
and/or technical know-how to carry on a comprehensive occupational 
competency test development program. Through the grant provided by 
the U.S. Office of Education and with the copperatioa of vocational 
teacher educators, state^upervisors of trade and Industrial education, 
examinations have been developed in twenty-four major occupations. 
Each examination consists of a written and a performance test. Two 
forms have been developed for each occupation. 
EVALUATION OF TEST RESULTS 

For the first time in vocational education these tests were 
administered through pilot testing programs in twenty-four test centers 
In nearly all regions of the country from a central location. The 
pilot testing program provided adequate data for statistical treatment 
of the results and test evaluation through item analysis and other ! 
appropriate measures. 
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While It is not possible within the limitations of this presentation 
to provide specific data for each teat perhaps some general comments 
about the quality of the tests might be of interest. The data for the 
written tests showed a reliability range from .882 to .925. The mean 
for item difficulty ranged from .480 to .0510 and item discrimination 
from .325 to .410. For a first effort these results are indeed 
promising. More Importantly, we now have Information which enable 
further test Improvement. The perfonuance test, more difficult to eval- 
uate, proved equally promising data. More subjective In nature than 
the written test, the observations of the candidates were made according 
to a predetermined evaluation scale. The examiners had to confine their 
judgments to predetermined Items on the evaluation scale. Spot inspection 
during performance tests revealed a high correlation of ratings among 
performance examiners. 

The pilot testing proved that the test quality was good. It pro- 
vided the practicality of administering centrally developed testa on a 
national scale. 
HOW CAN THE TESTS B E USED? 

The question may be raised as to the proper use of these tests/ 
Although orlRlnnlly developed to assist teacher educators and vocational 
administrators in the appraisal of candidates for admission to industrial 
teacher education programs, they provide objective evidence for occupa- 
tional competence for certification and for experience evaluation for 
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degree programs. The National Occupational Competency Testing 
Institute has been established on a permanent basis to provide the 
kind of testing service which has long been needed by the states. 
WHAT CONSTITUTES AN EXAMINATION PACKAGE? 

So far we have talked about the background, method , use, quality, 
and number of the tests. Now let^s briefly describe what constitutes 
a test package. 

The following materials are provided: 

1. A Bulletin for the candidate which describes In 
general terms the testing program, the basic 
eligibility requirements, and how he can register. 

2. A Scope of the Written and Performance Tests for 
each occupation which tells the candidate the 
content of the examination. 

3- A Manual for Administering Written and Perfor m ance 
Tests for examiners. 

4. Directions to the performance examiner with 
detailed instructions for the preparation of shops 
or laboratories — materials required , tools, 
machines, equipment, and facilities needed as well 
as performance evaluation directions. 

5. The Written Test 

6. The Performance Test 
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SUMMARY 

After many years of strenuous efforts » a National Occupational 
Competency Testing Institute has been established to assure effective, 
continuing service to vocational education* The Institute functions 
under the administrative direction of the Consortium of States for 
Occupational Competency Testing of Future Vocational Teachers and will 
render services determined by a representative board and thus maintain 
close contact with the needs of the ctates. 

The Institute has obtained contractual commltments^^omjtge^^^^^ 
Educational Testing Service to administer and score the NOCTI exam- 
inations nationally . ETS will report the test results, carry on re- 
search and employ its resources in the development of additional tests* 
ETS is a non-profit organization and the leader in test development, 
conduct ef^ test programs, and research in education as well as 

vocational qualifications. By taking this , act ion the Institute 

• ... 
combines the sophisticated test experience of ETS with the practical 

approaches developed during the research and development phase/: of the 

project. 

This is a breakthrough in occupational competency evaluation which 
will serve to improve vocational teacher competence, raise professional 
recognition of vocational teachers, and contribute to further improved 
vocational education. 



